Image transform method for obtaining expanded image data, image processing apparatus and image display device therefor

ABSTRACT

It is an object of the present invention to output a clear, expanded image wherein step-shapes or chain-shapes of oblique lines are reduced, distortion is eliminated and no constituent lines of fonts or graphics are missing. An image transform method, for transforming original input image data into image data expanded by a ratio represented by a rational number or an integer, comprises: a vertical and horizontal interpolation unit, for reducing correlation in the vertical and horizontal directions of interpolated image data linearly expanded from an original image data input unit, and for generating first expanded image data; an oblique interpolation unit, for performing linear interpolation, based on correlation with a target pixel constituting the original image data and neighboring pixels arranged in oblique directions, using the neighboring pixels to generate second expanded image data; and a final result generator for employing the first expanded image data and the second expanded image data to generate a final image.

FIELD OF THE INVENTION

The present invention relates to an image transform method forprocessing original input image data, and in particular, to an imagetransform method for reducing the step-shapes or chain-shapes of obliquelines, which appear when original input image data are expanded, and forcontrolling the distortion of constituent lines, and to an imageprocessing apparatus and an image display device therefor.

BACKGROUND ART

Conventionally, the nearest neighbor method and bilinear interpolationhave been used widely as interpolation methods to obtain a match betweena high definition liquid crystal display (LCD) and a lower resolutionpersonal computer (PC) screen. These interpolation methods are popularbecause only a small number of calculations is required for the realtime processing of a high resolution image. However, it is known thatthe nearest neighbor method contributes to an increase in distortion inan expanded image, and therefore, the bilinear interpolation method,whereby the rough shape of an original image can be expanded, is oftenemployed. But it is also true that bilinear interpolation tends to blurresultant images, an effect that is especially remarkable when bilinearinterpolation is employed for images displayed on an image displaydevice, such as an LCD, that does not have the low-pass characteristics.

The bilinear interpolation method is an example of the multirateprocessing theory called linear interpolation. And because of this,interpolation methods based on the general multirate processing theoryhave been proposed that use more pixels than the number employed forbilinear interpolation. For example, a multirate interpolation methodusing a table search method for a linear filter is disclosed in U.S.Pat. No. 5,410,616. Further, in U.S. Pat. No. 5,594,676, linear filtershaving different orders are prepared, and are changed in accordance withthe characteristics exhibited by peripheral pixels. In addition,according to reference document 1 (IEEE Signal Processing Magazine, Vol.16, No. 6, pp. 22-38, Nov. 1999), pertaining to the generalization ofthe interpolation method, except for a point for obtaining an efficientnumber of calculations, a linear interpolation method wherebyinformation loss can theoretically be minimized can be substantiallyprovided as an interpolation method that uses a third-order B-spline.Thus, it becomes apparent that image processing using the linearinterpolation method is approaching the limit of its effectiveness.

When the above method is employed for image interpolation, verticalcalculations and horizontal calculations are sequentially performed, andone pixel in a low-resolution image is transformed into a set ofrectangular pixel regions, so that the appearance of step-shapes orchain-shapes of oblique lines can not be prevented. In order to preventthe appearance of such shapes, it is considered effective to performinterpolation by employing a directional filter that uses peripheralpixels aligned in the direction in which an oblique line is extended.For example, in reference document 2 (IEEE Proceedings of the 1995International Conference on Acoustics, Speech, and Signal Processing,Vol. 4, pp. 2383-2386, May 1995), and in U.S. Pat. No. 5,991,463, theuse of the directional, linear interpolation methods are proposed.

These image expansion methods are required for a computer monitor and aprojector in order to provide a match between a flat panel display, suchas an LCD, which has a fixed pixel resolution, and an image produced bythe graphics card of a PC. However, when an expansion process for anoriginal image is performed vertically and horizontally, or in theopposite order, as with the conventional bilinear interpolation method,correlation is increased vertically and horizontally for the pixels thathave been interpolated. As a result, when an oblique line is expanded, apixel displayed on a low-resolution screen is transformed into a groupof rectangular pixel regions, and a step-shape or chain-shape of obliqueline appears. And even when a high accurate linear interpolation isperformed using a third order B-spline, the above phenomenon becomesever more remarkable as the expansion ratio is increased. When thisphenomenon appears on the display of a high definition LCD, it seems tothe user that the physical definition of the output screen differs fromthe actual definition.

In reference document 2, the differences in pixels are calculated forpredetermined directions, and the reciprocals of values obtained for allthe directions are added together to acquire the normalized weightcoefficient. Linear interpolation is performed for pixels dispersed ineach direction corresponding to each weight coefficient, and to providean expanded image, the summation of the results is calculated based onthe weight coefficients. In this method, linear interpolation isperformed by regarding, as the dominant direction, the one in which thepixel difference is small, and this method can be considered as a methodfor preventing the step-shapes or chain-shapes of oblique lines.However, using this method, since in order to prevent an erroneousdetermination the summation is calculated by also using theinterpolation results obtained for the non-dominant direction, blurringof an output image can not be avoided.

In U.S. Pat. No. 5,991,463, a bilinear interpolation technique usingfour adjacent pixels arranged in a specific direction is proposed as adirectional interpolation method. Specifically, according to thismethod, first, a difference among four peripheral pixels is obtained forthe determination of three directions, i.e., the right oblique, the leftoblique and the vertical directions. The differences are compared witheach other, and the direction having the smallest pixel difference isdetermined to be an interpolation direction. In addition, an additionalsearch is performed, as needed. When the pixel differences for the threedirections are substantially the same, the vertical direction isdetermined. When the pixel difference for the vertical direction issmall and the difference for one of the oblique directions is obviouslysmall, the pertinent oblique direction is determined. When the pixeldifferences for the two oblique directions are small, the same directiondetermination is performed for a pixel that forms an oblique lineparallel to the pixels whose difference was calculated. And when thedifferences for the two oblique directions are small for the additionalpixel, the vertical direction is determined. In this manner, as isdisclosed in this publication, when interpolation is performed for onespecific direction, excessive image blurring can be avoided.

However, according to the above method, an erroneous determination maybe made if in an observation pixel mask used for direction determinationforeground and background can not be identified, such as when white andblack pixels intersect to form an X shape. When the direction can not becorrectly determined, an interpolation value is obtained that is fardifferent from the correct value, and there is a defect in theinterpolation results. In addition, generally, according to theexpansion method for which only oblique interpolation is performed,correlation in the vertical and horizontal directions is reduced. Thatis, since at the point where vertical and horizontal linear linesintersect, an image is expanded as if it is assumed as a diamond shape,instead of expanding in the original vertical and horizontal directions,the obtained image is distorted. Further, since the isolated point ofthe original image is determined to be the point whereat the obliquelines cross, and the image is to be expanded in the oblique directions,the isolated point is faded out, or the expansion of the image is lessthan it is supposed to be.

To resolve the above shortcomings, it is one object of the presentinvention to provide an image transform method whereby the step-shapesor chain-shapes of oblique lines are not remarkable, even when an imagehas been expanded by two or more times of its original size.

It is another object of the present invention to output a clear,expanded image wherein distortion is eliminated and no constituent linesof fonts and graphics are missing. It is an additional object of thepresent invention to perform fast image transform processing.

SUMMARY OF THE INVENTION

To achieve the above objects, according to the present invention, basedon a linearly expanded image that is interpolated in the vertical andhorizontal directions, or vice versa, an intermediate process image isobtained wherein the vertical and horizontal correlation is reduced.Further, the direction correlated with the target pixel is detected, andthe image is expanded by using neighboring pixels arranged in thedetected direction to produce a resulting image. The processing of theintermediate image and the resulting image are employed to generate afinal, expanded image. Specifically, according to a first aspect of thepresent invention, an image transform method, for transforming originalinput image data into image data expanded by a ratio represented by arational number or an integer, comprises the steps of: reducingcorrelation in the vertical and horizontal directions of a linearlyexpanded image to generate first expanded image data; performing linearinterpolation, based on correlation with a target pixel constituting theoriginal image data and neighboring pixels arranged in obliquedirections, using the neighboring pixels to generate second expandedimage data; and employing the first expanded image data and the secondexpanded image data to generate a final image.

The step of generating the first expanded image data includes the stepsof: raster-scanning a window having a predetermined size wherein thetarget pixel and the neighboring pixels in the linearly expanded imagedata are included; and reducing vertical and horizontal directionalcorrelation through a rank order processing using median operation inthe window. This configuration is preferable because to perform theprocess for reducing the vertical and horizontal directionalcorrelations, simple operations can be used for which no enormous amountof calculation is required.

Further, the step of generating the second expanded image data includesthe steps of: determining an interpolation direction based on values ofdifferences between the target pixel and the neighboring pixels; andperforming linear interpolation in the determined direction. This stepleads to relatively precise determination of interpolation direction.

The image transform method further comprises the step of: regarding, asan adjustment value, the personal preference of a user concerning imagequality, wherein, at the step of generating the final image, based onthe adjustment value, the final image is generated by using the firstand the second expanded image data. As a result, an output image can beobtained by extracting suitable portions from the two sets of expandedimage data, e.g., by reducing a bulging shape that is exaggerated by thesecond expanded image data and appears at a crossing point.

According to a second aspect of the invention, an image transform methodcomprises the steps of: forming an image by linearly expanding originalimage data in the vertical and horizontal directions; and reducing thevertical and horizontal directional correlation of the image through arank order processing using, for example, median operation to generatean expanded image.

The image transform method further comprises the steps of: determining,for the expanded image, whether the contrast in the original image datacan be maintained at a predetermined level; and extracting a highfrequency component from the expanded image, when the contrast can notbe maintained at the predetermined level, and adding the frequencycomponent multiplied by a constant to the expanded image, or subtractingthe frequency component multiplied by a constant from the expandedimage. This configuration is preferable because the contrast can bemaintained at a predetermined level, even when, depending on how therank order processing is performed, the contrast in the original imagecan not be maintained. To determine whether the contrast is maintainedat the predetermined level, the contrast of the foreground and thebackground of the original image is compared with the contrast of thosesides of the expanded image, and the result is compared with apredetermined threshold value.

According to a third aspect of the invention, an image transform methodcomprises the steps of: reading a target pixel and neighboring pixels inoriginal image data; employing the target pixel and the neighboringpixels to calculate directional differences for the horizontal, thevertical, the right oblique and the left oblique directions; employingthe directional differences to determine a strong correlated direction;and performing linear interpolation for the target pixel using theneighboring pixels arranged in the strong correlated direction.

The image transform method further comprises the steps of: readingperipheral pixels arranged within a predetermined mask range adjacent tothe target pixel and/or the neighboring pixels; and accumulatingdifferences between the peripheral pixels, and between the target pixeland the neighboring pixels; and determining an interpolation direction,based on the cumulative value of the differences, and performinginterpolation in the interpolation direction. As a result, even for thinlines obtained by anti-aliasing (process for fusing the dots at bordersto the background), the direction of the foreground can be obtainedsubstantially correctly.

According to a fourth aspect of the invention, an image transform methodcomprises: an input step of entering original image data to be expandedby a magnification of two or more; a first process step of reducing thestep-shapes or chain-shapes of oblique lines appearing when the originalimage data are expanded by doubled or greater in size; a second processstep of expanding, in the oblique direction, the structure of theoriginal image data, and reducing a bulging shape appearing when aportion is expanded whereat vertical and horizontal lines of theoriginal image data cross each other; and an output step of outputtingan image expanded by the magnification of two or more using results ofthe first and second process steps.

Further, to achieve the above objects, according to a fifth aspect ofthe present invention, an image processing apparatus comprises: inputmeans for entering original image data to be expanded; vertical andhorizontal directional interpolation means for interpolating theoriginal image data in the vertical and horizontal directions; verticaland horizontal directional correlation reduction means for reducingcorrelation of the obtained image in the vertical and horizontaldirections; oblique direction detection means for detecting an obliquedirection having a strong correlation with a target pixel andneighboring pixels in the original image data; and directionalinterpolation means for employing the neighboring pixels in the detectedoblique direction to perform interpolation in the oblique direction.

The image processing apparatus further comprises: generation means forgenerating expanded image data based on an image obtained by thevertical and horizontal directional correlation reduction means and animage obtained by the oblique directional interpolation means. Thisconfiguration is preferable because the advantages offered by thesemeans facilitate the output of an image the user preference, forexample.

The image processing apparatus further comprises: input means forentering, as an adjustment value, the personal preference of a userconcerning image quality, wherein the generation means employs theadjustment value to synthesize the image obtained by the vertical andhorizontal directional correlation reduction means with the imageobtained by the oblique directional interpolation means. The adjustmentvalue entered at the input means need not be the one that is directlyused by the image processing apparatus, and a sensuous input value(e.g., setup due to comparison, such as softer or clearer, or a gradualsetup) relative to the personal preference of the user may betransformed for synthesization.

The vertical and horizontal directional correlation reduction meansperforms the ranked median value selection, such as the medianoperation, for the target pixel and the neighboring pixels in thelinearly expanded image data, and thereby reduces the correlation of animage in the vertical and horizontal direction. This configuration issuperior because a high quality, expanded image can be obtained by usinga small, simple processing apparatus.

Further, the oblique direction detection means employs differencesbetween the target pixel and the neighboring pixels to detect, withstrong correlation, the oblique direction, and the oblique directionalinterpolation means performs linear interpolation in the obliquedirection detected by the oblique direction detection means.

According to a sixth aspect of the invention, an image processingapparatus comprises: a vertical and horizontal linear interpolation unitfor forming an image by linearly expanding original image data in thevertical and horizontal directions; and a vertical and horizontaldirectional correlation reduction processing unit for reducing, for theimage, a vertical and horizontal directional correlation using a rankorder processing, and for generating an expanded image.

According to a seventh aspect of the invention, an image processingapparatus comprises: an interpolation direction determination unit forreading a target pixel and neighboring pixels in original image data,for calculating directional differences between the target pixel and theneighboring pixels for vertical, horizontal, right oblique and leftoblique directions, and for determining an interpolation direction basedon the directional differences; and an oblique directional linearinterpolation unit for performing linear interpolation for the targetpixel by using the neighboring pixels arranged in the determinedinterpolation direction. The interpolation direction determination unitreads peripheral pixels arranged within a predetermined mask rangeadjacent to the target pixel and/or the neighbor pixels and addstogether the differences between the peripheral pixels and the targetpixel and the neighbor pixels, and determines the interpolationdirection based on the cumulative value of the differences. Thisconfiguration is preferable because, when the interpolation directioncan not be identified using only the directional differences among thedirections, the interpolation direction can be accurately determined.

According to an eighth aspect of the present invention, an image displaydevice, for transforming low-resolution original color image data intohigh-resolution expanded color image data, and for outputting thehigh-resolution expanded color image data, comprises: first imageexpansion means for outputting an expanded image wherein the verticaland horizontal structure is maintained with reducing the step-shapes orchain-shapes of oblique lines in the linearly expanded image; secondimage expansion means for expanding the structure of the original colorimage data in the oblique direction, for reducing a bulging shape thatappears at intersections of lines, and for outputting an expanded image;and display means for employing the expanded images obtained by thefirst and the second image expansion means to display a final image.

The original color image data includes thin lines obtained byanti-aliasing, and the second image expansion means performsinterpolation, while correctly identifying the foreground based onpixels constituting the original thin lines, not based on pixelsobtained by anti-aliasing.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram showing an image processing apparatus according toone embodiment of the present invention.

FIG. 2 is a flowchart for explaining the processing performed by thevertical and horizontal interpolation unit 12.

FIGS. 3A and 3B are diagrams for explaining the reduction of astep-shape of boundary using the rank order processing.

FIG. 4 is a flowchart for explaining the processing performed by theoblique interpolation unit 15.

FIGS. 5A, 5B and 5C are diagrams showing example pixel patterns.

FIGS. 6D, 6E, 6F, 6G and 6H are diagrams showing example pixel patterns.

FIG. 7 is a flowchart for explaining directional search processing usinga cumulative difference.

FIGS. 8A and 8B are diagrams showing example pixel patterns for which acumulative difference is employed.

FIG. 9 is a flowchart for explaining the directional search processingusing a doubled difference.

FIGS. 10A, 10B and 10C are diagrams for explaining an oblique linebefore expansion and a final image obtained using the conventionalexpansion method.

FIGS. 11A and 11B are diagrams for explaining a final image obtained forthe embodiment.

FIGS. 12A and 12B are diagrams for explaining a crossed portion beforeexpansion and a final image obtained using the conventional expansionmethod.

FIG. 13 is a diagram for explaining a final image obtained for theembodiment.

FIGS. 14A and 14B are diagrams showing the state wherein an isolatedpoint is expanded.

DESCRIPTION OF THE SYMBOLS

11: Original image data input unit12: Vertical and horizontal interpolation unit13: Linear interpolation performed in the vertical and horizontaldirection order14: Vertical and horizontal directional correlation reduction process15: Oblique interpolation unit16: Interpolation direction determination17: Oblique linear interpolation18: Final result generator19: Final image output unit

DETAILED DESCRIPTION OF THE INVENTION Preferred Embodiment

The preferred embodiment of the present invention will now be describedin detail while referring to the accompanying drawings.

An image processing apparatus according to this embodiment is, forexample, a liquid crystal display monitor, which receives, from aconnected host apparatus, such as a PC, original image data consistingof R (red), G (green) and B (blue) color signals for a text screen or acomputer aided design (CAD) screen, and performs a resolution transformfor the image data, after which it outputs the results on a displayunit, such as a liquid crystal display cell. For an LCD monitor or anotebook PC, this liquid crystal display monitor can be implemented, forexample, by an internal LCD controller. The image processing apparatusis effective when, for example, original color image data generated by alow-resolution VGA (Video Graphics Array) (640×480 dots) graphic chip isdisplayed on a high-resolution UXGA (Ultra Extended Graphics Array)(1600×1200 dots) LCD display. Further, for a notebook PC the imageprocessing apparatus is implemented to include the host side. It shouldbe noted that no further detailed explanation will be given for a liquidcrystal display cell and the X and Y drivers that supply power to thecell.

In this embodiment, to improve the conventional method whereby one pixelin an original image is expanded as a rectangular group of pixels, andthe oblique lines of the original image assume a step shape or a chainshape in the resultant image, a method and an apparatus are proposed forgenerating a clear, expanded image wherein the structural balance of thecharacters or the graphic design of the original image is maintained.The factors for determining the adequateness of an expansion are thosethat are considered to be effective, so that a user feels natural to anexpanded image when he or she looks at a graphic design, such as a textscreen or a computer aided design screen, displayed on a high-definitionLCD. It should be noted that this method can also be applied when a userviews a natural image, and is not limited in use only to an expandedimage displayed on an LCD screen.

FIG. 1 is a diagram illustrating an image processing apparatus accordingto the embodiment of the present invention. The image processingapparatus roughly comprises: an original image data input unit 11, avertical and horizontal interpolation unit 12, an oblique interpolationunit 15, a final result generator 18 and a final image output unit 19.The original image data input unit 11 receives original image dataduring the process performed for each coordinate, and pixel values aresupplied thereto in a time-series manner as, for example, aone-dimensional sequential data stream that is received as a horizontalscan line. The vertical and horizontal interpolation unit 12 performs inthe named order, for the original image data received from the originalimage data input unit 11, a linear interpolation 13 process in thevertical and horizontal directions, and a vertical and horizontaldirectional correlation reduction process 14 for reducing the verticaland horizontal directional correlation. Thus, a large, step-shapeportion, acquired as a result of the linear interpolation 13 expansionprocess, is changed, and a smooth shape is produced by the vertical andhorizontal directional correlation reduction process 14 and issubsequently output. The oblique interpolation unit 15 performs, for theoriginal image data received from the original image data input unit 11,a process 16 for determining the interpolation direction and an obliquelinear interpolation 17. Thus, the oblique direction can be accuratelydetermined and the interpolation process can be performed in the obliquedirection.

The final result generator 18 synthesizes, at a predetermined ratio, theresults received from the vertical and horizontal interpolation unit 12and the oblique interpolation unit 15 in order to maintain the characteror graphic design balance. The predetermined ratio for thissynthesization can be obtained as an adjustment value for image qualitythat reflects the personal preference of a user. Further, thisadjustment value can be obtained, for example, by a user employing inputmeans (not shown) of a PC to enter data representing his or her reactionto the display of an expanded image. The final image output unit 19outputs the results obtained from the final result generator 18 to aliquid crystal display cell driver, for example, which displays theimage obtained by this embodiment on a liquid crystal display.

When a low-resolution PC screen image is to be expanded and displayed ona high-definition LCD, according to many expansion methods, an obliqueone pixel-width line in the original image is transformed into astep-shape or chain-shape line consisting of a rectangular group ofpixels. As the main factor in this case, it is assumed that for thecalculation dependence is to be placed on the sequential arrangement ofthe interpolated pixels, and a strong correlation, such as that in thevertical and horizontal directions, occurs among interpolated pixels.Whereas, if the calculation is performed by using peripheral pixelsarranged in the direction in which the step-shape or chain-shape lineappears, this problem may be avoided. However, at the same time, thevertical and horizontal correlation is reduced, and at the portion wherethe vertical and horizontal lines of a font cross each other, an obliquecrossing that differs from the original image appears, so that thebalance of the font is distorted, or at the point at which the linescross a large round dot will appear. Further, if an isolated point on anoriginal image is interpolated with weak correlation in the vertical andthe horizontal directions, the isolated point will be regarded as acrossing point for oblique lines, and will be expanded in an obliquedirection, so that it is faded and blurred. In this embodiment, in orderto avoid these phenomena, the final result generator 18 generates afinal expanded image using the vertical and horizontal interpolationresults, which are output by the vertical and horizontal interpolationunit 12 in order to maintain design balance, and the obliqueinterpolation results, which are output by the oblique interpolationunit 15 in order to reduce the step-shapes or chain-shapes of obliquelines.

FIG. 2 is a flowchart for explaining the processing performed by thevertical and horizontal interpolation unit 12. First, vertical andhorizontal bilinear interpolation is performed for the original imagedata received from the original image data input unit 11 (step 101). Thevertical and horizontal linear interpolation 13 described whilereferring to FIG. 1 can be performed using the conventional bilinearinterpolation method, and cubic convolutional interpolation orinterpolation using a B-spline can also be performed, so that theinterpolation method is not especially limited. Following this, as thevertical and horizontal directional correlation reduction process 14, atarget pixel and peripheral pixels are read (step 102) and sorted (step103), and the median value is determined as the pixel value at targetedcoordinates (step 104).

Specifically, the peripheral pixels that are included in a mask of aninteger matrix that does not exceed the expansion ratio (e.g., a mask oftwo rows and two columns if the expansion ratio is 2.5) are sorted inorder in accordance with value, and the median value is output as theprocess result. As the process for outputting the median value, a rankorder processing is used for outputting a predetermined order value, ora process for repetitively copying the pixels in the mask and sortingthem, and for performing a rank order processing for the sorted resultscan be employed. So long as the vertical and horizontal directionalcorrelation can be reduced, any acceptable arbitrary process can beemployed. Then, whether the current pixel is the last one is determined(step 105). If the pixel is not the last one, program control returns tostep 102 to repeat the above processing. If the current pixel is thelast one, program control advances to step 106.

At step 106, a check is performed to determine whether the contrast ofthe original image can be maintained through the above processing. Forthis determination, the difference in the luminances at a boundaryportion is obtained and is compared with a threshold value. When theluminance difference is smaller than the threshold value, it isascertained that the contrast can not be maintained. Whether or not theenhancement process is required depends on the employment of the rankorder, and when the contrast of the original image can be maintained,the pixel value determined at step 104 is output unchanged to the finalresult generator 18. But when it is determined the contrast of theoriginal image can not be maintained, the image enhancement process isperformed at steps 107 to 109. The process performed at steps 107 to 109is an image sharpening process called unsharp masking. During thisprocess, vertical and horizontal directional correlations are reduced atthe portion whereat the median value is output, and the high frequencycomponent of the image is enhanced in order to restore the imagecontrast obtained from the vertical and horizontal linear interpolation13. Specifically, the high frequency component is extracted from theimage (step 107), and the extracted image is multiplied by a constantvalue (step 108). The pre-extracted image and the multiplied image areadded together (step 109), and the result obtained at step 109 is outputto the final result generator 18.

As is described above, the vertical and horizontal interpolation unit 12can linearly expand original image data in the vertical and horizontaldirections, or vice versa, and can thereafter perform a non-linearprocess for changing the vertical and horizontal directional image thatis obtained to a set of non-directional correlation pixels. With thisconfiguration, a non-linearly expanded image, wherein the step-shapes orchain-shapes of oblique lines are reduced, can be output withoutdistorting the balance of the linear structure forming a character or agraphic in the original image, and without losing an isolated point inthe image.

FIGS. 3A and 3B are diagrams for explaining the reduction of astep-shape of boundary using the above described rank order processing.In FIG. 3A is shown the state obtained through the expansion process forwhich the vertical and horizontal linear interpolation 13 in FIG. 1 isused, and in FIG. 3B is shown the state obtained when the vertical andhorizontal directional correlation reduction process 14 in FIG. 1 isused. In FIGS. 3A and 3B, the pixel values in 49, 7×7 windows arerepresented as expansion results, with “32”, for example, being definedas bright and “0” being defined as dark. A thick solid line indicates aboundary line, and as is indicated by the thick solid line in FIG. 3A, aboundary having large steps occurs as the result of the expansionprocess for which only the vertical and horizontal linear interpolation13 is used.

In this embodiment, the rank order processing is performed after theexpansion process in FIG. 3A for which the vertical and horizontallinear interpolation 13 is used. Specifically, the rank order processingis performed by raster-scanning a window, enclosed by a broken line 21in FIG. 3A, in the direction indicated by an arrow 22, and the pixelpositions are adjusted in accordance with the even numbers of the pixelsin the window. In the area enclosed by the broken line 21, four windowsare defined by a pixel 23 at the second row and the second column, apre-scan pixel (second row and first column) and the pixels along thepreceding line (first row and first column and second column), and themedian operation, for example, is performed for the four windows todetermine the output value for the pixel 23 on the second row in thesecond column. The results obtained by the rank order processing areshown in FIG. 3B. The “*” symbols along the first row and along thefirst column represent “Don't care”, because the values of the pixelsbefore the scanning and along the preceding row are unknown.

The pixel, for example, on the second row in the sixth column in FIG. 3Ais “0”. When the median operation is performed for this pixel, thepre-scan pixel (second row and fifth column) is “32” and the pixelsalong the preceding row are “32” and “0”. When these pixels arerearranged in the ascending order, “0, 0, 32, 32”, the median value is“16”. As a result, as is shown in FIG. 3 b, the pixel on the second rowin the sixth column is transformed to “16”. Similarly, the pixel, forexample, on the third row in the fourth column in FIG. 3A is “0”, andthe vertical and horizontal pixel values in the window are rearranged inthe ascending order, “0, 32, 32, 32”. Therefore, the median value is“32”, and the output value is indicated on the third row in the fourthcolumn in FIG. 3B. When the rank order processing has been performed inthis manner, it is apparent that, as indicated by the thick solid linein FIG. 3B, the large steps are removed from the boundary, and further,the boundary is interpolated with the median values. As a result, aclear and smooth expanded image can be obtained at the currentresolution.

The processing performed by the vertical and horizontal interpolationunit 12 will now be described by using a general calculation example.

In this example, a calculation will be described for the processingincluding bilinear interpolation, the use of a median filter, and theunsharp masking for an image expansion of 2.5 times.

First, by using the following equation, the weighted mean using thedistance of interpolation value B_(k)[X] is performed for real numbercoordinates X between two adjacent pixels S[X_(k)] and S[X_(k+1)] in anoriginal image.

B _(k) [X]=sS[X _(k)]+(1−s)S[X _(k+1)],

for X _(k) ≦X≦X _(k+1) , s+X−X _(k)  [Equation 1]

Bilinear interpolation is the method used to perform this calculationfor the order in the vertical and horizontal directions.

When the expanded result obtained by bilinear interpolation is definedas B(x, y), and the size of the mask is 2×2, as previously described,the median filter employs the following equation for the processing.

M(x, y)=median{[B(x−1, y−1), B(x−1, y), B(x, y−1), B(x, y)]}  [Equation2]

In this example, when the number of vector elements of the target isodd, the median value for output is provided by median operation, orwhen the number of elements is even, the output value is determined inthe next process. For the above equation, when the pixel values arerearranged in the ascending order, as [R₁, R₂, R₃, R₄] (R₁≦R₂≦R₃≦R₄),the output provided by median operation is

M(x, y)=(R ₂ +R ₃)/2  [Equation 3]

When the output of an integer value is expected, a floor calculation forrounding off the calculation results in the negative infinite directionis performed, and the result

└M(x, y)┘  [Equation 4]

is employed as the output value. Further, for the extraction of a highfrequency component, simple means can be employed, such as asecond-order Volterra filter in the following equation proposed in theIEEE Transactions on Image Processing, Vol. 5, No. 6, pp. 950-963, June1996.

$\begin{matrix}{{V\left( {x,y} \right)} = {{2{M^{2}\left( {x,y} \right)}} - {{M\left( {x,{y - 1}} \right)} \cdot {M\left( {x,{y + 1}} \right)}} - {{M\left( {{x - 1},y} \right)} \cdot {M\left( {{x + 1},y} \right)}}}} & \left\lbrack {{Equation}\mspace{14mu} 5} \right\rbrack\end{matrix}$

In the same manner as in a method frequently used for unsharp masking, aspecific scalar value γ is provided to obtain the final results usingthe following equation.

N(x, y)=M(x, y)+γ·V(x, y)  [Equation 6]

In this case, instead of a Volterra filter, a Laplacian filter may beemployed for a high-pass filter, and the image enhancement method is notthereby limited.

As is described above, when the thus arranged vertical and horizontalinterpolation unit 12 in FIG. 1 is employed, a step-shape or chain-shapeof an oblique line can be reduced, while output with no distortion ismaintained in the vertical and horizontal crossed lines of a characterthat is obtained using common linear interpolation. However, since thisprocess is primarily for the rearrangement of the pixels in order toreduce the vertical and horizontal correlation, a problem of unevencoloration arises. Thus, in this embodiment, to resolve this problemusing the interpolation results wherein the step-shapes or chain-shapesof oblique lines and uneven coloration appear less frequently, theoblique interpolation unit 15 in FIG. 1 is provided, even though thevertically and horizontally crossed lines are distorted.

FIG. 4 is a flowchart for explaining the processing performed by theoblique interpolation unit 15. In this embodiment, basically, theinterpolation direction determination process 16 is performed, and thenthe oblique linear interpolation process 17 is performed, wherein pixelsin the interpolation direction are employed for linear interpolation.

First, the neighboring pixels at an inverse projected point of a targetexpansion point are obtained from the original image data input unit 11(step 201), and differences for the horizontal and the vertical, and theright and the left oblique directions are calculated (step 202).Specifically, when inverse projection is performed from the targetcoordinates obtained by expansion to the original coordinates, fourperipheral pixels are employed to calculate differences in thehorizontal and the vertical, and the right and the left obliquedirections. Then, the differences are compared with each other, i.e., acheck is performed to determine whether the horizontal or verticaldifference is equal to or smaller than a threshold value (step 203).

When the horizontal or the vertical difference is equal to or smallerthan the threshold value, a check is performed to determine whether thedifference in the left oblique direction is equal to or smaller than thethreshold value (step 204). And if the difference in the left obliquedirection is equal to or smaller than the threshold value, a check isperformed to determine whether the difference in the right obliquedirection is equal to or smaller than the threshold value (step 205).Then, should the difference be equal to or smaller than the thresholdvalue, i.e., when the difference in each of the two oblique directionsis equal to or smaller than the threshold value, the vertical andhorizontal directions are determined to be interpolation directions(step 210). Whereas if at step 205 the difference in the right obliquedirection is greater than the threshold value, the left obliquedirection is determined to be the interpolation direction (step 211). Ifat step 204 the left oblique direction is found to be greater than thethreshold value, a check is performed to determine whether thedifference in the right oblique direction is equal to or smaller thanthe threshold value (step 206). In this case, if the difference in theright oblique direction is equal to or smaller than the threshold value,the right oblique direction is determined to be the interpolationdirection (step 212). But if at step 206 the difference in the rightoblique direction is found to be greater than the threshold value, thenthe vertical and horizontal directions are determined to beinterpolation directions (step 213).

If at step 203 the difference in the horizontal or vertical direction isgreater than the threshold value, a check is performed to determinewhether the difference in the left oblique direction is equal to orsmaller than the threshold value (step 207). Then, when that differenceis equal to or smaller than the threshold value, a check is performed todetermine whether the right oblique difference is equal to or smallerthan the threshold value (step 208). And if the difference is equal toor smaller than the threshold value, i.e., when the difference in eachof the two oblique directions is equal to or smaller than the thresholdvalue, the directional search is performed using the cumulative value ofthe differences (step 214). But when at step 208 the difference in theright oblique direction is greater than the threshold value, the leftoblique direction is determined to be the interpolation direction (step215). Again, if at step 207 the difference in the left oblique directionis greater than the threshold value, a check is then performed todetermine whether the difference in the right oblique direction is equalto or smaller than the threshold value (step 209). And when thedifference in the right oblique direction is equal to or smaller thanthe threshold value, the right oblique direction is determined to be theinterpolation direction (step 216). If, however, at step 209 thedifference in the right oblique direction is greater than the thresholdvalue, the directional search is performed using a doubled difference(step 217). It should be noted that the threshold value used throughoutthis process is an arbitrary value, determined while consideration isgiven to the states of the pixels, and is stored in a memory (notshown).

Finally, based on a determination made at one of steps 210 to 217,linear interpolation is performed using pixels arranged in theinterpolation direction (step 218). Thereafter, the results obtained bylinear interpolation are output to the final result generator 18.

FIGS. 5A to 5C and 6D to 6H are diagrams showing example pixel patternsthat correspond to the respective cases described in FIG. 4. In thesediagrams, the relationship between the pixel values is conceptuallyrepresented, whose relationship consists of an inverse projected pointof a target expansion point and the neighboring pixels of the projectedpoint, and in this embodiment, bilinear interpolation is performed usingthese four neighboring pixels. The pixel pattern in FIG. 5A correspondsto step 210 in FIG. 4, the pixel pattern in FIG. 5B corresponds to step211, and the pixel pattern in FIG. 5C corresponds to step 212. Further,the pixel pattern in FIG. 6D corresponds to step 213, the pixel patternin FIG. 6E corresponds to step 215, the pixel pattern in FIG. 6Fcorresponds to step 216, the pixel pattern in FIG. 6G corresponds tostep 214, and the pixel pattern in FIG. 6H corresponds to step 217. Inthese pixel patterns, the circles represent pixels, and when the pixelvalues are near each other, i.e., when the difference between them isequal to or smaller than a threshold value, the pixels are connected bythick, solid lines.

According to the determination at step 210 in FIG. 4, the pixel patternin FIG. 5A is that found when the differences in all the horizontal andvertical, and right and left oblique directions are equal to or smallerthan the threshold value and are near each other. In this case, obliqueinterpolation need not be performed, and conventional interpolation isperformed in the vertical and horizontal directions.

When the difference in the vertical or horizontal direction is equal toor smaller than the threshold value, and when the difference in one ofthe oblique directions is also equal to or smaller than the thresholdvalue, the pertinent oblique direction is determined to be theinterpolation direction. In the other cases, the vertical and horizontaldirections are determined to be the interpolation directions (steps 211,212 and 213). The pixel pattern when the left oblique direction isdetermined to be the interpolation direction is shown in FIG. 5B; thepixel pattern when the right oblique direction is so determined is shownin FIG. 5C; and the pixel patterns in the other cases when the verticaland horizontal directions are so determined are shown in FIG. 6D. Whenthe difference in the horizontal or vertical direction is greater thanthe threshold value, the differences in the oblique directions arecompared as shown in the pixel patterns in FIGS. 6E and 6F, and thedirection having a value smaller than the threshold value is determinedto be the interpolation direction (steps 215 and 216).

At this time, the following two cases must also be coped with. In thefirst case, as is shown in FIG. 6G, the difference in the horizontal andvertical directions are equal to or greater than the threshold value,and the differences in the right and left oblique directions are equalto or smaller than the threshold value. Here, the differences betweenperipheral pixels that are adjacent to the diagonal lines connecting theabove four neighboring pixels are added together, and the cumulativevalue is employed to determine the interpolation direction (step 214 inFIG. 4).

FIG. 7 is a flowchart for explaining a directional search performedusing the cumulative value of differences. First, the pixels in a 6×6mask around the inverse projected point are read (step 301). Thedifference accumulation range is so defined that it is large enough tocover the essential structure, in the original image, of a font segmentof one pixel line. In many cases, a 6×6 mask range can provide asufficient window size that has two more peripheries than a 2×2 window.Here it should be noted, however, that no limit is imposed on the rangesthat can be used for masks.

The cumulative value DL of the pixel difference values for the leftoblique direction is obtained (step 302), as is the cumulative value DRof the pixel difference values for the right oblique direction isobtained (step 303). Thereafter, a check is performed to determinewhether the absolute value for (DL-DR) is equal to or smaller than athreshold value (step 304). When the absolute value is equal to orsmaller than the threshold value, the vertical and the horizontaldirections are determined to be the interpolation directions (step 305).When at step 304 the absolute value for (DL-DR) is greater than thethreshold value, the cumulative values DL and DR are compared (step306). If the cumulative value DL, for the pixel difference values in theleft oblique direction, is greater, the left direction is determined tobe the interpolation direction (step 307). Whereas when the cumulativevalue DR, for the pixel difference values in the right obliquedirection, is greater, the right direction is determined to be theinterpolation direction (step 308). In this manner, the side having thesmaller cumulative value is determined to be the background, and theopposite diagonal direction is identified as the interpolationdirection. When the difference between the cumulative value for theright oblique direction and the cumulative value for the left obliquedirection is equal to or smaller than a threshold value, the verticaland horizontal directions are determined to be the interpolationdirections.

FIGS. 8A and 8B are diagrams showing an example pixel pattern for whichthe cumulative value of the differences is employed. The pixel patternof a portion of the alphabetical character “a”, enclosed by a thin linein FIG. 8A, is shown in FIG. 8B. As is shown in FIG. 8B, the crossing ofthe foreground and background in a small font has a line width of onedot. When the dot dimensions 6×6 are used to view such a font, it isfound that the number of pixels used for the foreground of the font issmaller than the number of pixels used for the background. Therefore,the simplest way to identify the foreground is to compare the number oflike colored pixels. However, for thin lines obtained, for example, byanti-aliasing (the process used for fusing the dots at the boundary withthe background), if the thin lines cross, due to the anti-aliasing, thecolor of the pixels at the boundary is mixed with the color used on thebackground, and the number of pixels having mixed colors is smaller thanthe number of originally colored pixels on the foreground. In thisembodiment, therefore, for each color, color differences are accumulatedfor an entire area within a dimensionally defined space. Thus, since thecolor of pixels obtained by anti-aliasing is closer to the color of thebackground, the accumulation difference decreases, and the color usedfor the foreground is correctly detected as the color having the largestaccumulation difference.

An explanation will now be given for the second case that must be copedwith. In the second case, as is shown by the pixel pattern in FIG. 6H,each of the differences in the horizontal and vertical directions and inthe right and left oblique directions are greater than the thresholdvalue (step 217 in FIG. 4). In this case, a directional search using adoubled difference is performed.

FIG. 9 is a flowchart for explaining a directional search performedusing a doubled difference. For this process, first, a horizontaldifference h, a vertical difference v, a left oblique difference dl anda right oblique difference dr are read (step 401). Then, the process isbranched, depending on whether the absolute value of a differencebetween the oblique differences is equal to or smaller than thethreshold value, i.e., a check is performed to determine whether theabsolute value of (dl-dr) is equal to or smaller than the thresholdvalue (step 402). If the absolute value is equal to or smaller than thethreshold value, the interpolation direction is determined to be thevertical and the horizontal directions (step 403). But if the absolutevalue is greater than the threshold value, the differences in the rightand left oblique directions are compared (step 404).

When at step 404 dl<dr, i.e., when the difference dl in the left obliquedirection is smaller, a check is performed to determine whether h<dl orv<dl is satisfied between the left oblique difference and either thehorizontal difference or the vertical difference (step 405). When h<dlor v<dl, the interpolation direction is determined to be the verticaland horizontal directions (step 407). When h<dl or v<dl is notsatisfied, i.e., the difference in the left oblique direction issmaller, the left oblique direction is determined to be theinterpolation direction (step 408).

Similarly, when dl<dr is not satisfied at step 404, i.e., when thedifference dr in the right oblique direction is smaller, a check isperformed to determine whether h<dr or v<dr is satisfied between theright oblique difference and the horizontal difference or the verticaldifference (step 406). When h<dr or v<dr, the interpolation direction isdetermined to be vertical and the horizontal directions (step 409). Butif h<dr or v<dr is not satisfied, i.e., when the right obliquedifference is smaller, the right oblique direction is determined to bethe interpolation direction (step 410).

Thus, the same linear interpolation as the conventional bilinearinterpolation is performed by using four neighboring pixels arranged inthe thus determined interpolation direction and at the inverse projectedpoint. It should be noted that when the pixels in the oblique directionare employed, the distance in the oblique direction can be employed asthe distance between the original image pixel and the interpolationpixel used for bilinear interpolation. Further, when the distances alongthe vertical and horizontal directions are employed, because of thesimilarity in the relationship established for the distance ratio,substantially the same results can be obtained as when an obliquedirection is employed. Note that the number of directional divisionsused in the above explanation is only an example, and that anothernumber may be employed.

As is described above, since, in this embodiment, unlike in U.S. Pat.5,991,463, the oblique interpolation unit 15 determines the directionusing the cumulative difference values, it can be expected thatreliability will be statistically improved and that the number oferroneous determinations will be reduced. As is understood from thepixel selection method, the process performed by the obliqueinterpolation unit 15 renders the step-shapes or chain-shapes of obliquelines substantially not remarkable. Further, since the reduction of thecorrelation, which is caused by the vertical and horizontal directionalcorrelation reduction process performed by the vertical and horizontalinterpolation unit 12, does not occur, uneven coloration is notremarkable in an output image expanded by a multiple of two or more.However, according to this method, the pixels in the oblique directionsare employed for a Chinese (Kanji) character wherein many lines crossvertically and horizontally. Thus, the balance of the image obtained byexpanding a character or graphic included in the original image aredistorted; a large round dot (a bulging shape), for example, appears atcrossing points in the original image; and following the imageexpansion, an isolated point is small and blurred.

Therefor, in this embodiment, the non-linear interpolation resultobtained by the vertical and horizontal interpolation unit 12, and theresult obtained by the oblique interpolation unit 15 are employedtogether, so that the disadvantages of the two offset each other andtheir advantages are established. The main advantage of the non-linearinterpolation result obtained by the vertical and horizontalinterpolation unit 12 is that the balance of characters and graphics ismaintained. The advantage of the interpolation result obtained by theoblique interpolation unit 15 is a reduction in step-shapes orchain-shapes of oblique lines and in uneven coloration. As an example ofthe generation of an image for which these two advantages are provided,when an image obtained using non-linear interpolation is represented byN(x,y) and an image obtained using oblique interpolation is representedby S(x,y), a final image R(x,y) can be obtained by using the adjustmentvalue α.

R(x,y)=αN(x,y)+(1-α)S(x,y)  [Equation 7]

Since the image for which the two advantages are provided is generatedby using the adjustment value α, it is possible to obtain a clear finalimage wherein a step-shape or chain-shape of oblique line or unevencoloration in the same color area is not noticeable, and the balance ofthe lines composing a character or graphics in an original image is notdistorted. As is described above, the adjustment value α can bedetermined from an entry made by a consonant with his or her personalpreference. Further, the user entry need not be the value α, and basedon the predetermined value set by the user, the adjustment value may betransformed internally by the image processing apparatus. As an examplesetup by a user, a sensuous expression, such as softness or clarity, canbe designated using a number of levels by employing a display or apointing device.

FIGS. 10 to 14 are diagrams for explaining the final image in thisembodiment when an expansion of 2.5 times is employed. An oblique linebefore expansion is shown in FIG. 10A, and the result obtained by theexpansion of the oblique line in FIG. 10A using common bilinearinterpolation is shown in FIG. 10B. Next, in FIG. 10C, the result isshown that is obtained by interpolating the oblique line in FIG. 10Ausing a third-order B-spline. The common bilinear interpolation shown inFIG. 10B is the expansion method currently equipped in many LCDmonitors, and it is apparent that when this method is used to expand animage, the step-shape or chain-shape of oblique line is remarkable.Since the interpolation for which the third-order B-spline in FIG. 10Cis used, is regarded as a method for obtaining a precision near thetheoretical upper limit that can be achieved by linear interpolation,this method is shown for comparison. It is apparent, however, that evenwhen interpolation using a third-order B-spline is employed, step-shapesor chain-shapes of oblique lines are remarkable.

FIG. 11A is a diagram showing the expanded state obtained, by thevertical and horizontal interpolation unit 12 of this embodiment, merelythrough non-linear interpolation. Due to the use of non-linearinterpolation, the step-shapes or chain-shapes of oblique lines arereduced; however, changes in luminance, the cause of uneven coloration,occur in a few places. FIG. 11B is a diagram showing the final image ofthe oblique line that is output in this embodiment, while taking intoconsideration the result obtained by the oblique interpolation unit 15.As is described above, according to the embodiment, the number ofstep-shapes or chain-shapes of oblique lines, which occur during theexpansion of an oblique image, is reduced, and an expanded image can beobtained wherein uneven coloration is less remarkable.

FIG. 12A is a diagram before expansion of an example pattern in whichthere is a crossed portion. While FIG. 12B is a diagram showing thepattern of a final image provided by the oblique interpolation unit 15of this embodiment. Since the oblique interpolation unit 15 determinesthat the crossed portion is continued obliquely, as is shown in FIG.12B, a phenomenon occurs whereby a round dot having a bulging shape islocated at the crossed portion. In this embodiment, since as is shown inFIG. 13 both the vertical and horizontal interpolation unit 12 and theoblique interpolation unit 15 are employed to generate a final result,the above problem can also be resolved.

FIGS. 14A and 14B are diagrams showing the state wherein an isolatedpoint is expanded. If an isolated point, represented in the originalimage by a single pixel, is expanded in the oblique direction byemploying the method using pixels arranged in the direction, the pointis faded out, as is shown in FIG. 14A. However, since in the embodimentnon-linear interpolation in the vertical and horizontal directions istaken into account, an expanded image can be obtained without unduefading of the isolated point occurring.

As is described above, according to this method, it is possible toobtain a final, clear image wherein the number of step-shapes orchain-shapes of oblique lines is reduced and uneven coloration in thesame color area is not remarkable, and the balance of the linescomposing a character or graphics in an original image is not distorted.

As is described above in detail, according to the embodiment, thevertical and horizontal interpolation unit 12 is employed, so that theoriginal input image data are linearly expanded in the vertical andhorizontal directions, or vice versa, and non-linear interpolation isperformed for an expanded image having the vertical and horizontaldirectional correlations to obtain a set of pixels having no directionalcorrelation. Thus, a non-linear expanded image can be obtained whereinstep-shapes or chain-shapes of oblique lines are reduced, withoutdistorting the balance of the line structure of characters or graphicsin an original image, or losing an isolated point.

Furthermore, the oblique interpolation unit 15 is employed, so that thetarget pixel value in the original image data and the neighboring pixelvalues of the target pixel are used to detect connection relationshipswith the target pixel. Then, based on the obtained results, pixelselection is performed and the direction in which pixels are arranged isdetermined to perform linear interpolation. Following this, theexpansion process is performed through linear interpolation usingperipheral pixels arranged in oblique directions. Because of this pixelselection method, the occurrence of step-shapes or chain-shapes ofoblique lines can be reduced.

Furthermore, the final result generator 18 generates an image, using apredetermined adjustment value α, based on the first image obtained bythe vertical and horizontal interpolation unit 12 and the second imageobtained by the oblique interpolation unit 15. Thus, a resolution can beprovided for a preferable transformed image that includes the advantagesprovided for the first and second images.

As is described above, according to the present invention, theoccurrence of step-shapes or chain-shapes of oblique lines is reduced,and the distortion or the absence of lines constituting a font orgraphics is also reduced, so that a clear, expanded image can be output.

The present invention can be realized in hardware, software, or acombination of hardware and software. The present invention can berealized in a centralized fashion in one computer system, or in adistributed fashion where different elements are spread across severalinterconnected computer systems. Any kind of computer system—or otherapparatus adapted for carrying out the methods described herein—issuitable. A typical combination of hardware and software could be ageneral purpose computer system with a computer program that, when beingloaded and executed, controls the computer system such that it carriesout the methods described herein. The present invention can also beembedded in a computer program product, which comprises all the featuresenabling the implementation of the methods described herein, andwhich—when loaded in a computer system—is able to carry out thesemethods.

Computer program means or computer program in the present context meanany expression, in any language, code or notation, of a set ofinstructions intended to cause a system having an information processingcapability to perform a particular function either directly or afterconversion to another language, code or notation and/or reproduction ina different material form.

It is noted that the foregoing has outlined some of the more pertinentobjects and embodiments of the present invention. This invention may beused for many applications. Thus, although the description is made forparticular arrangements and methods, the intent and concept of theinvention is suitable and applicable to other arrangements andapplications. It will be clear to those skilled in the art that othermodifications to the disclosed embodiments can be effected withoutdeparting from the spirit and scope of the invention. The describedembodiments ought to be construed to be merely illustrative of some ofthe more prominent features and applications of the invention. Otherbeneficial results can be realized by applying the disclosed inventionin a different manner or modifying the invention in ways known to thosefamiliar with the art.

1-4. (canceled)
 5. An image transform method, for transforming originalinput image data into image data expanded by a ratio represented by arational number or an integer, comprising the steps of: forming an imageby linearly expanding original image data in the vertical and horizontaldirections; and reducing the vertical and horizontal directionalcorrelation of said image through a rank order processing to generate afinal expanded image, wherein the rank order processing includes:raster-scanning a window enclosing a target pixel and one or more of itsneighboring pixels; and computing an output value of the target pixel byperforming an averaging operation on pixels enclosed within the window.6. The image transform method according to claim 5, further comprisingthe steps of: determining, for said expanded image, whether the contrastin said original image data can be maintained at a predetermined level;and extracting a high frequency component from said expanded image, whensaid contrast can not be maintained at said predetermined level, andadding said frequency component multiplied by a constant to saidexpanded image, or subtracting said frequency component multiplied by aconstant from said expanded image. 7-14. (canceled)
 15. An imageprocessing apparatus for transforming original input image data intoexpanded image data comprising: a vertical and horizontal directionallinear interpolation unit for forming an image by linearly expandingoriginal image data in the vertical and horizontal directions; and avertical and horizontal directional correlation reduction processingunit for reducing, for said image, a vertical and horizontal directionalcorrelation using a rank order processing to generate a final expandedimage, operating in combination for transforming the original inputimage data into expanded image data, wherein the rank order processingincludes: raster-scanning a window enclosing a target pixel and one ormore of its neighboring pixels; and computing an output value of thetarget pixel by performing an averaging operation on pixels enclosedwithin the window.
 16. An image processing apparatus for transformingoriginal input image data into expanded image data comprising: aninterpolation direction determination unit for reading a target pixeland neighboring pixels in original image data, and for determining aninterpolation direction, wherein the neighboring pixels comprise a firstneighboring pixel and a second neighboring pixel, and whereindetermining an interpolation direction comprises: calculating a leftoblique difference using the target pixel and the first neighboringpixel; calculating a right oblique difference using the target pixel andthe second neighboring pixel; determining the left oblique direction tobe the interpolation direction when the left oblique difference issmaller than a threshold value and when the right oblique difference isgreater than a threshold value; and determining the right obliquedirection to be the interpolation direction when the left obliquedifference is greater than a threshold value and when the right obliquedifference is smaller than a threshold value; and an oblique directionallinear interpolation unit for performing linear interpolation for saidtarget pixel by using said neighboring pixels arranged in saiddetermined interpolation direction, operating in combination fortransforming the original input image data into expanded image data. 17.The image processing apparatus according to claim 16, wherein saidinterpolation direction determination unit reads peripheral pixelsarranged within a predetermined mask range adjacent to said target pixeland/or said neighbor pixels and adds together the differences betweensaid peripheral pixels and said target pixel and said neighbor pixels,and determines said interpolation direction based on the cumulativevalue of said differences. 18-20. (canceled)
 21. An article ofmanufacture comprising a computer usable medium having computer readableprogram code means embodied therein for causing image transformation,the computer readable program code means in said article of manufacturecomprising computer readable program code means for causing a computerto effect the steps of claim
 5. 22-24. (canceled)
 25. A program storagedevice readable by machine, tangibly embodying a program of instructionsexecutable by the machine to perform method steps for imagetransformation, said method steps comprising the steps of claim 5.26-28. (canceled)
 29. A computer program product comprising a computerusable medium having computer readable program code means embeddedtherein for causing image processing, the computer readable program codemeans in said computer program product comprising computer readableprogram code means for causing a computer to effect the steps of claim15.
 30. A computer program product comprising a computer usable mediumhaving computer readable program code means embedded therein for causingimage processing, the computer readable program code means in saidcomputer program product comprising computer readable program code meansfor causing a computer to effect the steps of claim
 16. 31. (canceled)